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Abstract. We consider open billiard flows in IR" and show that the standard symplectic form da 
in IR" satisfies a specific non-integrability condition over their non- wandering sets A. This allows 
to use the main result in [St3] and obtain Dolgopyat type estimates for spectra of Ruelle transfer 
operators under simpler conditions. We also describe a class of open billiard fiows in IR" (n > 3) 
satisfying a certain pinching condition, which in turn implies that the (un)stable laminations over 
the non- wandering set are C^. 

1 Introduction 

It is well-known that hyperbolic billiard fiows in compact domains (e.g. Sinai billiards in Euclidean 
spaces or on tori) are non-integrable, just like contact Anosov flows (see e.g. [KB] or Appendix 
B in [L]). However, when considering contact flows over basic sets A the general non-degeneracy 
of the contact form (which implies the non-integrability) does not say much about the dynamics 
of the flow over A. It is much more natural, and it turns out to be important as well, to look at 
the restriction of the contact form over tangent vectors to A (see Sect. 2 for the deflnition). This 
is what we do here for non-wandering sets of open billiard flows. The motivation to study this 
kind of non-integrability comes from [St3] which deals with spectral estimates of Ruelle transfer 
operators for flows on basic sets (see Sect. 6 below for some details). 

Let i^T be a subset of IR" (n > 2) of the form K = Ki U K2 U . . . U Kk^ , where Ki are compact 
strictly convex disjoint domains in IR" with boundaries Fj = dKi and > 3. Set 0, = IR" \ K 
and r = dK. We assume that K satisfles the following (no-eclipse) condition: 



With this condition, the billiard flow (pt defined on the sphere bundle S{Q) in the standard way is 
called an open billiard flow. It has singularities, however its restriction to the non-wandering set 
A has only simple discontinuities at refiection points. Moreover, A is compact, (f)t is hyperbolic 
and transitive on A, and it follows from |Stl) that (j)t is non-lattice and therefore by a result of 
Bowen [B], it is topologically weak-mixing on A. 

Our main aim in this paper is to show that the open billiard fiow always satisfies a certain 
non-integrability condition on A. Let da be the standard symplectic form on T(IR") = IR" x IR". 

Theorem 1.1. There exist zq £ A and /_i > such that for any unit tangent vector b G E^{zq) to 
A there exists a unit tangent vector a G E''{zq) to A with \da{a,b)\ > /x. 

If the map A 3 x ^ E^{x) is C^, then the invariance of da along the flow implies that the 
points zq £ A with the above property form an open and dense subset of A. Theorem 1.1 is 
established by means of a certain pairing of points on the strong stable and unstable manifolds of 
an appropriately chosen point zq - see Sect. 3 and Lemma 3.1 there for details. As a consequence 




for every pair iTj, Kj of different connected components of K the convex hull of 
Ki U Kj has no common points with any other connected component of K. 
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of this and the main result in [St3j one gets Dolgopyat type spectral estimates for pinched open 
billiard flows - see Sect. 6. 

It is well-known that in general the maps A B x E^{x) (or E^[x)) are only Holder continuous 
(see e.g |HPS| or |PSWj ). The following pinching condition implies stronger regularity properties 
of these maps. 

(P): There exist constants C > and < a < /3 such that for every x G A we have 
1 



e"-^ limi < ||#t(2;) • n|| < Ce'^"* ||u|| , nGS"(x),t>0 
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for some constants ax ,(ix > depending on x hut independent of u and t with a < ax < Px ^ P 
and 2ax — Px > o- for all x £ A. 

For example in the case of contact flows (f)t, it follows from the results in [Ha2j (see also |Halj ) 
that assuming (P), the map A 9 x i— t- E^{x) is C^"*"^ with e = 2q//3 — 1 > (in the sense that this 
map has a linearization at any x G A that depends Holder continuously on x). The same applies 
to the map A3 x ^ E''{x). 

Notice that when n = 2 (then the local unstable manifolds are one-dimensional) this condition 
is always satisfied. It turns out that for n > 3 the condition (P) is always satisfied when the 
minimal distance between distinct connected components of K is relatively large compared to 
the maximal sectional curvature of dK (see Proposition 1.2 below). An analogue of the latter 
for manifolds M of strictly negative curvature would be to require that the sectional curvature is 
between —Kq and —aKo for some constants Kq > and a £ (0, 1). It follows from the arguments 
in [HPj that when a = 1/4 the geodesic flow on M satisfies the pinching condition (P). 

Set dij = dist{Ki, Kj) and do = mindjj. Since every Ki is strictly convex, the operator 

Lx : Tx{dK) — > Tx{dK), LxU = (VuZ^)(2;), of the second fundamental form is positive definite 
with respect to the outward unit normal field i^iy), y £ dK. Then k{x, u) = {LxU, u) is the normal 
curvature of dK at x in the direction of u G Tx{dK), \\u\\ = 1. Set 

Kmin = min mill {Lx{u),u) , Kmax = max max {Lx{u),u) , 

x€dK ueT^{dK),\\u\\=l x&dK u£T^(dK),\\u\\=l 

where (•, •) is the standard inner product in IR". 

Before continuing, notice that the condition (H) implies the existence of a global constant 
<y9o G (0,7r/2) such that for any x G A and any reflection point q of the billiard trajectory 7(x) 
generated by x the angle (/? between the reflected direction of 7(x) at q and the outward normal 

to dK at q satisfies < ^po- Set i^o = 2 cos ipo Kmin and Aq = -; — I • 

do cos ipo 

Let a > be such that dij < do + a for all i, j = 1, . . . , k^, i ^ j . Below we assume that do is 
large compared to a and Kmaxi so that 

(1.1) [1 + {do + a) Xo]'''^'' <{l + do fio?''" . 

(Notice that when a = r do, < r < 1, then the above holds for all sufficiently large do, assuming 
Kmax and Kmin are uniformly bounded above and below, respectively, by positive constants.) 
In Section 5 below we prove the following 

Proposition 1.2. Assume that (1.1) holds and the boundary dK is C^. Then the open billiard 
flow (/)t in the exterior of K satisfies the condition (P) on its non-wandering set A. Moreover, for 
any x £ A we can choose ax = ao and Px = Po; where ao = '"'''^^^^"^^"^ o,nd j3o = ^'^(^+('^^+") , 
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This is relatively easy to derive from a formula for the growth of the differential of the flow 
on unstable manifolds (see Proposition 5.1). The latter can be proved using an argument similar 
to that in the Appendix in [St2j (dealing with the two-dimensional case), and also can be easily 
derived from more general facts about the evolution of unstable vectors for multidimensional 
dispersing billiards (see e.g. [BCST]). 

Section 2 below contains some basic definitions and an example which concerns the geometry 
of the non- wandering set A. Sections 3 and 4 are devoted to the proof of Theorem 1.1. In 
Section 5 we use some well known formulae of Sinai for curvature operators related to unstable 
manifolds of dispersing billiards to prove Proposition 1.2. Section 6 deals with Dolgopayt type 
estimates for pinched open billiard flows - these are straightforward consequences of [St3j and the 
considerations in Sect. 3 below. 

Acknowledgement. The author is grateful to the referee whose comments and criticism led to a 
significant improvement of the first version of the paper. 

2 Preliminaries 
2.1 Basic definitions 

Let M be a complete Riemann manifold, and 0j : M — > M {t £ IR) a fiow on M. A 
0t-invariant closed subset A of M is called hyperbolic if A contains no fixed points and there 
exist constants C > and < A < 1 such that there exists a d^f-invariant decomposition T^M = 
E^{x)®E''{x)®E%x) ofT^M {x G A) into a direct sum of non-zero linear subspaces, where E^{x) 
is the one-dimensional subspace determined by the direction of the flow at x, \\d(j)t{u)\\ < C A* ||u|| 
for all u e E'{x) and t > 0, and ||#t(n)|| < CA"* ||u|| for all u G E'^ix) and t < 0. 

A non-empty compact (/)t-invariant hyperbolic subset A of M which is not a single closed orbit 
is called a basic set for (pt if is transitive on A and A is locally maximal, i.e. there exists an 
open neighbourhood 1/ of A in M such that A = DteR't^tiy)- 

For X £ A and a sufficiently small e > let 

W:ix) ={yeM: diM^), Mv)) < e for ah i > , diM^), Mv)) -^t^oo } , 

W^'ix) = {yGM: d{<l)t{x) At{y)) < e for ah t < , d{ct>t{x) At{y)) ^t->-oo } 

be the (strong) stable and unstable manifolds of size e. Then E^{x) = TxW^{x) and E^{x) = 
T^W^ix). Given z G A, let exp^ : ^"(z) W^^^{z) and exp| : E'{z) W/^(z) be the 
corresponding exponential maps. A vector b E E'^{z) \ {0} is called tangent to A at z if there exist 
infinite sequences {w^™)} C E'^iz) and C IR\{0} such that exp^(tm v(")) G AnW^{z) for ah 
m, v^"^^ — >• b and tm —i- as m oo. It is easy to see that a vector b G E'^{z) \ {0} is tangent to A 
at z iff there exists a curve z{t), < t < a, in W^{z) for some a > with z{0) = z, z{0) = b, 
and z{tn) G A for some sequence {tn}^=i C (0, a] with t„ — )• as n — )• oo. Tangent vectors to A in 
E^{z) are defined similarly. Denote by E^[z) (resp. E^{z)) the set of all vectors b G E'^{z) \ {0} 
(resp. b G E^[z) \ {0} ) tangent to A at z. 

Remark 1. Although we have not sought to construct particular examples, it appears that in 
general the set of unit tangent vectors to A does not have to be closed in the bundle E^ (or E^^. 
That is, there may exist a point z G A, a sequence {z^} C W^{z) n A and for each m a unit 
vector tangent to A at Zm such that z^ ^ z and ^ S, as m — )• oo, however ^ is not tangent 
to A at z. 
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Next, assume that K and il. are as in Sect. 1. The non-wandering set A for the flow (pt is the 
set of those x S S{Q,) such that the trajectory {(ptix) : t £ R} is bounded. Notice that the natural 
projection of (pt on the quotient space S{^1)/ ~, where ~ is the equivalence relation (g, v) ~ (p, w) 
iS q = p and v = woiq = p€ dK and v and w are symmetric with respect to Tq{dK), is 
continuous. Moreover whenever both x and (j)t{-c) are in the interior of S{i}) and sufficiently close 
to A, the map y i— )• 4>t{y) is smooth on a neighbourhood of x. It follows from results of Sinai 
( |Sil| . |Si2| ) that A is a hyperbolic set for (pt, and it is easily seen that A is the maximal compact 
(/)j-invariant subset of S{Q). Moreover, it follows from the natural symbolic coding for the natural 
section of the flow (the so called billiard ball map) that the periodic points are dense in A, and (pt 
is transitive on A. Thus, A is a basic set for (pt and the classical theory of hyperbolic flows applies 
in the case under consideration (see e.g. Part 4 in |KH] ). 

2.2 An example 

Here we briefly describe a (non-trivial) example from |St4| which shows that in general for every 
z G A the space span(ii^^(2;)) generated by the vectors in E'^{z) tangent to A could be a proper 
subspace of E^{z). 

Example 2.1. ( |St4| ) Assume that n = 3 and there exists a plane a such that each of the 
domains Kj is symmetric with respect to a. Setting K' = K ^^ a and Vt' = Vt f^ a, it is easy 
to observe that every billiard trajectory generated by a point in A is entirely contained in a. 
That is, A = A', where A' is the non-wandering set for the open billiard flow in il'. Thus, 
dim( span(£']((z))) = 1 < dim(i?"(z)) = 2 for any z G A. This example is of course trivial, since 
A is contained in the flow-invariant submanifold S*{Vl') of S*{Vt). 

However with a small local perturbation of the boundary dK of K we can get a non-trivial 
example. Choosing standard cartesian coordinates x,y,z in IR^, we may assume that a is given 
by the equation z = 0, i.e. a = IR^ x {0}. Let pr^ : S*{R^) ~ IR^ x — > IR^ be the natural 
projection, and let C = pr^(A). We may choose the coordinates x,y in the plane a = {z = 0} 
so that the line y = is tangent to K'l and K2 and K' is contained in the half-plane y > 0. Let 
qi E K[ and q2 G K2 be such that [(71,(72] is the shortest segment connecting K[ and K!^- Take 
a point q'l £ dK[ close to qi and such that the y-coordinate of y[ is smaller than that of gi. 
Consider the open arc A on dK[ connecting qi and q'l- It is clear that ACi C = ^. 

Let / : IR^ ^ IR^ be a (we can make it even C°°) diffeomorphism with f{x) = x for 
all X outside a small open set U such that qi £ U and U n dK' C A. Then for any q £ C the 
tangent planes Tq{dK) and Tq{dK) coincide. We can choose / so that Ki = f{Ki) = Ki for i > 1, 
Ki = f{Ki) is strictly convex, and v{f{q)) ^ a for q £ A arbitrarily close to qi. Here v is the 
outward unit normal field to dK. 

One can then show that the non-wandering set A for the billiard flow (pt in the closure of 
the exterior of K in IR^ coincides with A ( |St4j ). Thus, dim( span(^|(2;))) = 1 < <lmi{E^{z)) 
for any z £ A. However, it is clear from the construction that S* [a H J7) is not invariant with 
respect to the billiard flow (pf Moreover, it is not difficult to see that there is no two-dimensional 
submanifold a oi such that S*{a) is (i(/)j-invariant and A C S*{a)] see Section 4 in |St4] for 
details. 

3 Non-integrability of open billiard flows 

In this section we prove Theorem 1.1. 
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Let K C IR" be as in Sect. 1. For any x £ T = dK we will denote by z^(x) the outward unit 
normal to T at x. Given 6 > denote by Ss{^) the set of those (x,n) S S{Cl) such that there 
exist y GT and t > with y + tu = x, y + su £ IR" \ K for all s G (0, t) and {u, z^r(y)) ^ 

Remark 2. Notice that the condition (H) implies the existence of a constant 5o > depending only 
on the obstacle K such that any (x, u) € S{^) whose backward and forward billiard trajectories 
both have a common point with F belongs to S'^^ 

For e G (0,(io/2) set 

= {x = G : dist(g,aK) > e} , A, = AnPe(^^)- 

In what follows in order to avoid ambiguity and unnecessary complications we will consider stable 
and unstable manifolds only for points x in Pe(r2) or A^; this will be enough for our purposes. 
Fix for a moment arbitrary e, b and A so that 

(3.1) < (5 < e < A < ^ . 

We will see later how small these numbers need to be. 

Consider an arbitrary point do = (x^*^^ , ^^''•') G A^ such that z^^^ = x*-*^^ + A^*-''-' G dK, 
^(0) = -i/(z(o)) and x(°) + t^^^^ G IR" \ if for aU t G [0, A). I.e. the billiard trajectory generated 
by (To is perpendicular to dK at z^^^ and so the reflected direction at z^'^^ is Notice that 

there exist such point^, e.g. we can take x^*^^ on the shortest segment between two connected 
components Ki and Kj {i ^ j) with ^^^^ parallel to that segment. The local submanifolds 
U = W^{ao) and S = wfiao) have the form 

U = {{x,iyx{x)):xeX} , S = {{y , uy (y)) : y Y} 

for some smooth local (n — l)-dimensional submanifolds X and Y in IR", where i^x and i^y are 
continuous unit normal fields on X and Y. Moreover the second fundamental form L^-^^ of X 
with respect to ux (resp. L^^) of Y with respect to vy) is positive (resp. negative) definite. 
Finally, we have x*-''-' £ Xr\Y, X and Y are tangent at x^'^^ and vxix^'^^) = vy{x^'^'^) = Since 
the tangent planes to X and Y at xq are parallel to the tangent plane to dK at z^'^\ we have 

r,(o)X = r,(o)y = r^(o)(aif). 

Consider the inversion i : S(Q) — > S(Q) defined by i(x,^) = (x,— ^). It follows from the 
general properties of stable (unstable) manifolds that for any (x,^) G S (or U) sufficiently close 
to (To we have i o 02a(x, ^ U i^i respectively). In other words the shift along the billiard flow (^t 
of the convex front S along the normal field i'y{y) in 2A units coincides locally with the inversion 
of the convex front X near x^'^^ Using Sinai's formula ( |Si2| . cf. also [SiChj ) in the particular 
situation considered here we have 



/ + A B^(o) (u) 



where 

B^(o) (n) = ^ + 2L^{o) (n) , n G T^mX 



l + XL%](u) 



^In fact, it is not difficult to see that the union of the orbits of such points ctq is a dense subset of A. 
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It is well-known (see |Si2] ) that the curvature operators of strong unstable manifolds of (j)t are 
uniformly bounded, so there exists a global constant C > such that 2L^{o) (n) < B^{o) (n) < C 
for all u E T^{o)X, \\u\\ < 1. Therefore, 

(3.2) C'<lII]{u)<C , n G r,(o)X, 11^.11 < 1 , 

for some other global constant C" > (depending on K but not an A and u). 

Consider the map <1> : U — > S near (Tq = (x^'^^^^'')) defined by ^(x,,^) = i o 02a(2;,^). In 
fact by the same formula (see below for more details) one defines $ as a local smooth map 
$ : r(IR") = IR" X IR" — > r(IR") near ctq. Given e > 0, we will assume 6 G (0,e] is chosen 
sufficiently small , so that ^ is well-defined and ^(U) C S. Moreover, ^{z) = z' implies ^{z') = z 
(whenever $(z') is defined) and locally ^{W^{z)) = W^{z'). Finally, it is important to remark 
that preserves the set A. Indeed, (j)2x{^) = A and i(A) = A, as well. So, in particular 

(3.3) $(f/nA)cSnA. 

To write down a more explicit expression for <I>, let / be a defining function for dK in a 
neighbourhood of dK in IR" so that ||V/|| = 1 near z^'^^ and V/(z) = z^(z) is the outward unit 
normal to dK at z e dK. Then dK = /"^O) (locally near z(o)). Given (x,^) G IR" x IR" 
close to (To, there exist a unique z{x,^) G dK and a unique minimal t(x,^) G IR+ with z{x,^) = 
X + t{x,^)^ G dK , i.e. such that 

(3.4) f{x + t{x,m = 0- 

By r]{x,^) we denote the reflection of ^ with respect to V/(z(x,^)), i.e. 

r?(x,0 = C-2(e,V/(z))V/(z) . 

Here and in what follows we denote for brevity z = z{x, ^). We will also use the notation t = t{x, ^) 
and 7? = T]{x,^). We then have 

where g{x,S,) = z + (2A — t)r] . Since ^{U) C S and $ is a local diffeomorphism between U and 
S, we have (i$o-(-E"(cT)) = E^{a) for every a £ U. Moreover, it is easy to see that d$o- preserves 
the sets of tangent vectors to A, namely if cr G C/ n A and ^ G £'"(fT) \ {0} is tangent to A at a, 
then d^a^ ■ ^ is tangent to A at ^"(0"). 

It is well known that we can take the constant C > so large that < C for any a £ U 

and any choice of ctq (see e.g. [Si2) . [Chlj or [BCSTj ). (See also the proof of Lemma 3.1 in Sect. 

4 for an explicit formula for d^„g.) 

(X) 

Set L = L^(o) ^'^d H = L^(o), and for any u G T^n))X, consider the vectors 
v{u) = {u, Lu) G -E"(cro) , w{u) = (i$(fTo) • v{u) G £""(0-0) . 
It is easy to see that 

(3.5) \\v{u)\\ < v/l + C2||n|| . 

The following lemma is the main technical ingredient in the proofs of Theorem 1.1 and Propo- 
sition 6.2. Its proof is given in Sect. 4 below. 

Lemma 3.1. For any u,u' G T^(o)X we have da{v{u),w{u')) = {u,Pu') , where the linear 
operator P is given by P = 2H + 2L + 2\{HL + LH + L'^ + \ LHL) . Consequently, if k > is the 
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minimal principal curvature at a point on dK and e and A are chosen sufficiently small, then P 
is positive definite, {u,Pu) > k \\u\\^ , and therefore \da{v{u) ,w{u))\ > k H-up for all u G T^{o)X. 

Proof of Theorem 1.1. Take e > sufficiently small and then A with (3.1) small enough so that 
the operator P in Lemma 3.1 is positive definite, where ctq is chosen as above. (Notice that H and 
L are uniformly bounded from below and above regardless of the choice of the point gq.) More 
precisely, as stated in Lemma 3.1, if k > is the minimal principal curvature at a point on dK^ 
we can choose < e < A so small that («, Pu) > n \\u\\^ for any u G T^(o)X. 

Set zq = (To and let b G E^(zo), \\b\\ = 1. Then b = v{u) for some u G T^(o)X. Moreover, by 
(3.3), w{u) = d^{ao) ■ v{u) G Ef^{zo), and so a = i(;('u)/||u;(u)|| is a unit vector in Ej^{zo). By 
Lemma 3.1, 

1 II Il2 II Il2 II Il2 

' k\\u\\ h\\u\\ k\\u\\ 



\da{a,b)\ = j — \da{v{u),w{u))\ > -r — > 



w{u)\\ ' V V /> V //I - - (7111,(^)11 c 



On the other hand (3.5) implies 1 = ||6|| = [[^(tf)!! < vT+~C^ ||tt||, so \da{a,b)\ > cJT+c^- * 

As a consequence of Lemma 3.1 one can also derive the following which however we do not 
need in this paper. 

Proposition 3.2. For every z G A and every 5 > there exists 5 G A n Wg{z) such that for 
any non-zero tangent vector b G E'^{z) to A there exists a tangent vector a G E^{z) to A with 
dala, b) / 0. 

4 Proof of Lemma 3.1 

We will use the notation from Sect. 3. Recall that the standard symplectic form da has the form 

da{ {u, u) , {p,p)) = -{u,p) + {u,p) , 

where {u,u),{p,p) G T*(IR"). Given u,u' G T^mX, let v{u) = {u,u) G -E"(a), v{u') = {u',u') G 
E'^ia) and w{u') = d<I>^„(t;(u')) = {p,p). Then 

P\ = ( 9x9{cr) d(.g{a) \ fu'\^f d^gia) u' + d^g{a) u' 
p) y-d^Tjia) -d^r]{a) J \u' J \-dxV{cr) u' - d^ri{a) u' 

and so 

(4.1) da{v{u),w{u')) = {u, dxri{a)u') + {u, d^ri{a)u') 

+{ u , dxg{a) u') + {u, d^g{a) v! ) . 

One needs the derivatives of g and rj. Differentiating (3.4) gives 

(4.2) V,t(ao) = V/(z(°)) = -eW , V^i(c7o) = AV/(z(°)) = -AC(°). 
Moreover, z = x -\-t^ implies 

/ . „N dzf , ^ dt , ^. ^ dzf , ^ dt , ^ 

(4.3) _^(a;,^) = 5^.,+_(x,oe£ , ^(^'0 = % + ^(^,oe£- 
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Next, we have g{x, ^) = z + {2X - t) r] = x + 2X^ - 2{2X - t) V/(z)) V/(z) . Hence 

. ,,,2|-(..0«.V/(.),|^,.)-2,2A-*)(|_a^(.)||)|^(.) 
-2(2A-*)(«.V/(.))|:^(.)||. 

|l(x,f) = 2M, + 2 * (f. V/(.))gw - 2(2A - t) (.) 
Similarly, 



-^«.v/w)|:^(^)||. 

Notice that for u = {u,u) G E'^(ao) we have £t ± ^o- Therefore (4.2) imply {Vxticro) , tt) 
( V^t((To) , u) = 0, and the same holds with u replaced by u. This and (4.3) give 

j=i j j=i ?i 

for any u G T^{o)X. Using these, i(o"o) = A, the above formulae and the Hessian matrix 



H' =(J^{z^^))\ , one gets: 



= v!i-2X{u', //'e W ) 1^ (^(0) ) + 2A iH'u')i = u[ + 2X {H'u')i , 

OXi 

where {H'u')i is the ith coordinate of the (column) vector H'u' . Here we used the fact that 
^(0) = _v/(z(o)) and H'Vf{z^^^) = 0, since ||V/|| = 1 near dK. Similarly, 
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= + 2A (i^'wOi • 

The last four formulae imply (n, dxTjicro) u' ) = J27,j=i'^i'^j ^i'^o) = 2(u, H'u') , and sim- 
ilarly (ti, d^ri{ao)u') = {u,u') + 2\{u, H'u'), {u, dxg{(7o) u' ) = {u,u') + 2\{u, H'u') , and 
( « , d^g{ao) u' ) = 2X{u, u') + 2X'^{u, H'u'). Combining these with (4.1), one gets 

da{v{u),w{u')) = 2{u, H'u )+{u,u')+2X{u, H'u')+{u,u)+2X{u , H'u )+2X{u,u')+2X'^{u, H'u') . 

Using u = L{u), u' = L{u') and the fact that H'u = Hu for all u G T^(o)X, it now follows that 

da{v{u),wiu')) = 2{u, Hu') + {u,Lu') + 2X{u,HLu') + {Lu,u') + 2X{Lu, Hu') 
+2X{Lu,Lu') +2X'^{Lu,HLu') = {u,Pu') , 

where P = 2H + 2L + 2X {HL + LH + + X LHL). ■ 

5 Pinched open billiard flows 

In this section we describe some open billiard flows in IR" (n > 3) that satisfy the pinching 
condition (P). (Clearly open billiards in IR^ always satisfy this condition.) As one can see below, 
the estimates we use are rather crude, so one would expect that with more sophisticated methods 
larger classes of open billiard flows could be shown to satisfy the condition (P). 

First, we derive a formula which is useful in getting estimates for \\d(j)t{x) ■ u\\ (u £ E'^{x), 
X € A), both from above and below. From the arguments in this section one can also derive a 
representation for the Jacobi fields along a billiard trajectory. 

In what follows we use the notation from the beginning of Sect. 3. Here we assume that 
the boundary dK is at least smooth. 

Fix for a moment a point xq = {qo,vo) £ A^. If e > is sufficiently small, then W^{xo) has 
the form (cf. |Sil) . 18121 ) W^{xo) = {{x,i^xix)) ■ x G X} for some smooth hypersurface X in IR" 
containing the point qo such that X is strictly convex with respect to the unit normal field i^x- 
Denote by : TqX — > TqX the curvature operator {second fundamental form) X at q £ X. 
Then Bx is positive definite with respect to the normal field i^x (lSi2]). 

Given a point q G X, let 7(3;) be the forward billiard trajectory generated by x = {q,i^xiq))- 
Let qi{x),q2{x), ... be the refiection points of this trajectory and let Cj(x) G $"~^ be the reflected 
direction of 7(2;) at qj{x). Set qQ{x) = q, tQ{x) = and denote by ti{x),t2{x), . . . the times of the 
consecutive refiections of the trajectory 7(2;) at dK. Then tj{x) = do{x) + di{x) + . . . + dj^i{x), 
where dj{x) = \\qj^i{x) — qj{x)\\ , 1 < J . Given t > 0, denote by ut{q) the shift of q along 
the trajectory j{x) after time t. Set Xt = {ut{q) : q G X} . When ut{q) is not a refiection point 
of j{x), then locally near ut{q), Xt is a smooth convex (n — l)-dimensional surface in IR" with 
"outward" unit normal given by the direction vt{q) of j{x) at ut{q) (cf. [Si2]). 

Fix for a moment t > such that tm(a^o) < t < tm+iixo) for some m > 1, and assume 
that q{s), < s < a, is a curve on X with q{0) = qo such that for every s G [0, a] we have 
tm{x{s)) < t < tm+i{x{s)), where x(s) = {q{s),i'x{q{s))). Assume also that a > is so small 
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that for all j = 1,2,... ,m the reflection points qj{s) = qj{x{s)) belong to the same boundary 
component dKi^ for every s £ [0,a]. 

We will now estimate ||(i(/>t(xo) • Coll) where = ^(O) G Tg^X. 

Clearly (j)t{x{s)) = {p{s),Vmix{s))), where p{s), < s < a, is a curve on Xt- For brevity 
denote by 7(5) the forward billiard trajectory generated by {q{s),h'{q{s))) and set qo{s) = q{s). 
Let Cj(s) G be the reflected direction of 7(5) at qj{s) and let ipj{s) be the angle between 

Cj(s) and the outward unit normal v{qj{s)) of dK at qj{s). Let (j)t{x{s)) = {ut{s),vt{s)), and 
let tj{s) = tj{x{s)) be the times of the consecutive reflections of the trajectory 7(5) at dK. Set 
dj{s) = dj{x{s)) = \\qj+i{s)-qj{s)\\ {0 < j < m-l), to{s) = 0, im+i(s) = t and dm{s) = t-tm{s). 
Denote by kt{s) the normal curvature of Xt at ut{s) in the direction of -^utis). 

Next, let fco('S) be the normal curvature of X at q{s) in the direction of q{s), and for j > 
let A:j(s) > be the normal curvature of (s) = limt\t (s) at (?j(s) in the direction Uj{s) 
{\\uj{s)\\ = 1) of limtv,(s) For j > let 

be the curvature operator (second fundamental form) of -'^tj (s) at qj{s), and define ij{s) > by 
(5.1) [1 + dj{s)ij{s)f = l + 2djis)kj{s) + idjis)f \\Bj{s)uj{s)f . 

Finally, set 

Proposition 5.1. For all s G [0, a] w;e have 

(5.3) ||4o(5)|| = ||p(s)||^o(5)'5i(s)...'5m(5). 



As we mentioned in the Introduction, the above formula can be easily derived from the more 
general study of the evolution of unstable fronts in multidimensional dispersing billiards in [BCST] 
(see Section 5 there). Apart from that, one could prove (5.3) by using a simple modification of the 
argument in the Appendix of [St2j dealing with the two-dimensional case. We omit the details. 

We will now use Proposition 5.1 to prove Proposition 1.2. 

In the notation above, let qj = qj{x) be the reflection points of the billiard trajectory 7(2;) 
for some x = {q,vx{q))-, with q £ X, and let tj = tj{x) and dj = dj{x). Consider the curvature 
operator Bj = Eg. : Ilj = Tg.{Xt-) — > Tij , and let Sj : IR" — > IR" be the symmetry with 
respect to the tangent space Tj = Tg^{dK); notice that Sj{Ilj-i) = Hj. Let Nj : Tj — > Tj be the 
curvature operator (second fundamental form) of dK at qj. 

Notice that Ilj is the hyperplane in IR" passing through qj and orthogonal to ^j = ,(a,)(x); 
it will be identified with the (n — l)-dimensional vector subspace of IR" orthogonal to (,j. 

Before going on we need to recall the representation of the operator Bj due to Sinai [Si2] (cf. 
also Chernov |Chl] ). Introduce the linear maps Vj : Ilj — > Tj, V* : Tj — > Hj where Vj is (the 
restriction to Ilj of) the projection to Tj along the vector ^j, while V^* is the projection to Ilj 
along the normal vector Uj = v{qj). (Considering Vj : IR" — > Tj and V^* : IR"" — > Ilj, Vj* is the 
self-adjoint of Vj.) Let cpj be the angle between Vj and ^j. Then ( |Si2] ) 

(5.4) Bj = Sj Br Sj + 2 cos ipj V*NjVj 

for 1 < j < m, BJ = Bj^i {I+dj^i Bj^i)^^ , and Bm+i = Bm {I+f Bm)~^ , where t' = t-tm > e. 



10 



Let Hjixo) < Xj{xo) be the minimal and the maximal eigenvalues of the operator Bj. If A is 
an eigenvalue of -Bj_i, then A/(l + dj^iX) is an eigenvalue of , and ^ = i/x+d- i ^ IE ' 

Next, a simple calculation shows that the spectrum of the operator VjNjVj lies in the interval 
[«min, -^^V Thus, using (5.4) we get 

(5.5) iiQ < 2 cos (fj Kmin < IJ-j{xo) < Aj(xo) < J- + ^^^2^ < Ao , 

ciq cos 

where /xq and Aq are as in Sect. 1. 

Proof of Proposition 1.2. Before we continue, notice that there exist global constants < ci < C2 
such that ci ||^|| < \\u\\ < C2 ||^|| for any u = {^,r]) G E^{x), x G A (see formula (3.5) above). 

Assume that (1.1) holds. Fix an arbitrary xq = {qo,vo) G Ag and t > 0. We will now use the 
notation from the beginning of this section. 

To estimate \\d(pt{xo) - uW for a given unit vector u = rj) G E'^(xq), consider a curve q{s), 
< s < a, on X = pri(M^"(a;o)) with q{0) = qq and q(0) = 4, and define qj{s), j = 1, . . . ,m and 
p{s) as in the beginning of this section. Thenp(s) = pTi{d(j)t{x{s))), so ci||p(0)|| < ||ci(;l!)t(a;o) ■ u|| < 
C2||^?(0)||. Using this and Proposition 5.1, we get 

(5-6) ^7?^^?M^^-^^ < \\dM^o) ■ u\\ < "'"'''I 



C2Si{0)S2{0)...Sm{0) - " ""^ " - ci(5i(0)(52(0)...,5^(0) 

Recall that each 5j is given by (5.2) and (5.1), so if < lJ,j{xo) < \j{xo) are the minimal and 
maximal eigenvalues of the operator Bj{0), then 

(1 + d,{0)i,{0)f < 1 + 2d,(0)A, + 4(0) X] = (1 + dj{0) Xj{xo)f , 

so ij{0) < Aj(xo). Similarly, ^j(xo) < ^j(O). Moreover, it follows from (5.5) that /xq < fJ-jixo) and 
Xj{xo) < Xq for all j > 1 and all xq G A. 

Assuming \\u\\ = 1 and recalling that dj{xo) = dj{0) and t > di{xo) + . . . + dm{xo), (5.6) and 
(5.2) give 



-^ln\\dM^o)-u\\ < 1^1^^^^ + 1 In(l + djixo) A,(xo)) 



m 



^ ln(c2/ci) ^ Er=iMl + (^o + Q)Ao) 



di(a;o) + ■ ■ ■ + dm(aJo) 



^ ln(c2/ci) _^ m ln(l + (do + a) Aq) 



t mA 







ln(c2/ci) ln(l + (do + a) Ao) ln(c2/ci) 
S 7 1 = 1. r Po , 



so ||#t(xo) • u\\ < (c2/ci) e*^o for all t > 0. 

In a similar way, using (5.6) one derives that \\d(f)t{xo) ■ u\\ > d (ci/c2)e*"^° for i > 0, where 
"-^0 ~ '°^d^4-°a^°'* ^'^d c' > is another global constant. Finally, notice that (1.1) implies 2ao > 
Po + o. for some global constant a > 0. Hence the condition (P) is satisfied. ■ 
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6 Dolgopyat type estimates for pinched open billiard flows 



Let (pt : M — > M be a flow on complete (not necessarily compact) Riemann manifold M, 
and let A be a basic set for (p^. It follows from the hyperbolicity of A that if e > is sufficiently 
small, there exists 5 > Q such that if x,y G A and d{x,y) < 6, then W^{x) and (p^^^^^jiW^ (y)) 
intersect at exactly one point [x,y] G A (cf. |KHj ) . That is, there exists a unique t G [— e, e] such 
that M[x,y]) ^W:'{y). 

Let TZ = {Ri}^^i be a Markov family for (pt over A consisting of rectangles Ri = [Ui, Si], where 
Ui (resp. Si) are (admissible) subsets of W^{zi) n A (resp. W^{zi) n A) for some e > and Zi £ A 
(cf. e.g. |PPj for details; see also [D]). The first return time function t : R = u'^^iRi — > [0,oo) 
and the standard Poincare map V : R — > R are Lipschitz when restricted to an appropriate large 
subset of R. Set U = U^^^C/j and define the shift map a : U — > U hy a = poV, where p : R — > U 
is the projection along the leaves of local stable manifolds. Let U be the set of all u £ U whose 
orbits do not have common points with the boundary of R (in A). Given a Lispchitz function 
(or map) on U, we will identify it with its (unique) Lipschitz extension to U. Assuming that the 
local stable and unstable laminations over A are Lipschitz, the map a is essentially Lipschitz on 
U in the sense that there exists a constant L > such that if x,y E UiCi a~^{Uj) for some i,j, 
then d{a{x),a{y)) < Ld{x,y). The same applies to r : C/ — > IR. 

Given a Lipschitz real- valued function / on [/, set g = Qf = f — Pt, where P = £ IR is 
the unique number such that the topological pressure Pro- (5) of g with respect to a is zero (cf. 
e.g. [PP]). For a,b£ IR, one defines the Ruelle operator Lg_^a+ib)T ■ C^^'^iU) — > CLiP(C/) in the 
usual way (cf. e.g. |PPj or ^), where C^'P{U) is the space of Lipschitz functions g : U — >(L. 
By Lip ((7) we denote the Lipschitz constant of g and by ||g||o the standard sup norm of g on U. 

We will say that the Ruelle transfer operators related to the function f on U are eventually 
contracting if for every e > there exist constants < p < 1, ao > and C > such that if 
a, 6 G IR are such that \a\ < oq and |6| > 1/ao, then for every integer m > and every h E C^^^(JJ) 
we have 

\\Lj_^P^^^^,,)Mup,b < C p"^ \b\^ \\h\\up,,, 

where the norm ||.||Lip.6 on C^^P([/) is defined by ||/i||Lip,b = ll^llo + ^'j^j^^ . This implies in 
particular that the spectral radius of i^/-(p^.+a+ib)T C^^P(C/) does not exceed p. 

Next, assume that (pt is a contact flow on M with a invariant contact form to. The 
following condition says that du is in some sense non-degenerate on A near some of its points: 

(ND): There exist zq £ A, 6q > and pq > such that for any S G (0, 6q], any z £ ACi W^{zq) 
and any unit vector b £ E'^(z) tangent to A at z there exist z £ An W^(z), y £ W^{z) and a unit 
vector a £ E^{y) tangent to A at y with 

(6.1) \duj^{az,bz)\ > po 

where bz is the parallel translate of b along the geodesic in W^^{z) from z to z, while Oz is the 
parallel translate of a along the geodesic in W^^{z) from y to z. 

Remark 3. In fact, it is clear from the proof of Proposition 6.1 in |St3j that in (ND) the 'parallel 
translation' in the deflnition of the vector hz can be replaced by any other uniformly continuous 
(linear) operator Pz^z '■ E^{z) — > E^{z) {z,z £ An W^^{zo)). E.g. using a local coordinate 
system to 'identify' E'"'{z) and E'^(z) would be good enough. The same applies to the 'parallel 
translation' in the definition of the vector a^. In the case of the open billiard considered in this 
paper, (6.1) can be replaced simply by \da{a,b)\ > ^o- 
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As an immediate consequence of the main resulcl in [St3j (see also Sect. 6 there) one gets the 
following: 

Theorem 6.1.([St3]) Let (pt ■ M — > M be a contact flow on a Riemann manifold M and 
let A be a basic set for 4>t such that the conditions (P) and (ND) are satisfied for the restriction of 
the flow on A. Then for any Lipschitz real-valued function f on U the Ruelle transfer operators 
related to f are eventually contracting. 

Notice that for open billiard flows both W^{x) n A and W^{x) n A are Cantor sets, i.e. they 
are infinite compact totally disconnected sets without isolated points. In this particular case we 
can always choose the Markov family TZ = {Ri}^^i so that the boundary (in A) of each rectangle 
Ri is empty and therefore U = U. 

Next, assume that K is as in Sect. 1. Let (pt be the open billiard flow in the exterior of K 
and let A be its non- wandering set. 

The following consequence of Lemma 3.1 shows that under some regularity condition, the 
billiard flow satisfies the condition (ND) on A. 

Proposition 6.2. Assume that the map A 3 x t-^ E^{x) is . Then there exist G A, (5o > 
and Ho > such that for any 6 G iO,6o], any z £ ACi Wg^{zQ) and any unit vector b £ E'^{z) 
tangent to A at z there exist y S W^{z) and a unit vector a £ E^{y) tangent to A at y with 
\da{az,b)\ > /io, where Oz is the parallel translate of a along the geodesic in W^^^{s) from y to z. 

Proof of Proposition 6. 2. Notice that the standard symplectic form da in IR^" satisfies 

|da(e,r/)| < lieilhll , e,??GlR'", 

where we use the standard norm || • || in IR^". 

Assume that the map A 9 x i— E'^{x) is C"^; then the map A 9 x i— )• E^{x) is C^, as well. Fix 
(To as in Sect. 3, set zq = do, and choosing eo > and 5'q G (0,eo] sufficiently small, define the 
map 

^,Wl{zo)^W!^{z,) 

as in Sect. 3. It follows from Lemma 3.1 that there exists a constant /^i > (e.g. take = 
k/\/1 + C^) such that \da{d<^{zQ) ■b,b)\ > fii for all unit vectors b G E'^{zo). Take (5o > so small 
that 

(6.2) \da{d<^{z).b,b)\>^ , zGW^^{zo),beE''^{z),\\b\\ = l. 

Further restrictions on 6q will be imposed later. 

Next, assuming 6'q £ (0, €o] is sufficiently small, for any x £ A and y £ ACi W^, (x) the local 
holonomy map 'Hx,y '■ ^ H W^, (x) — > A n W^^ (y) along unstable laminations is well-defined and 
uniformly Holder continuous (see e.g. |HPSj or |PSW| ). Recall that the map is defined as 
follows. Given x' G A n Wy (x), there exist a unique y' G W^^{y) such that (ptiv') £ ^eoi^') 
some t G IR, |t| < eo- Then we set 'H'^y{x') = y' . Under the additional condition that the unstable 
laminations are C^, the maps 'IVf.y are as well (see e.g. Fact (2) on p. 647 in |Halj ). That 
is, for each x' G A n W^i{x) the map 'H'^y has a linearization L^y{x') : E^{x') — > E^{y') at 
x' G Aniy|, (x) and ||L",^(x')|| < Ci for some constant Ci > independent of x, y and x' . Notice 
that L^^y{x') preserves the sets of tangent vectors to A, namely if ^ G E^{x') \ {0} is tangent to 
A at x', then L^y{x') ■ is tangent to A at y' . 

^In fact, the main result in |St3] is much more general, however we are not going to discuss it here. 
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Since the map L^^^^^{x) depends continuously on z S W^i {zq) n A and x G W^, {zq) n A and 
-^"o 20 (-^o) = I (the identity operator), we can take (5q > so smah that 

(6.3) ||L^„,,(x)-/||<^ 

for all z G A n W^, (zq) and x G A n W|, {zq). We will assume (5q > is chosen so small that for all 
z G W^i (zo) n A, y G W^, {z) fl A and unit vectors a G E^{y) we have ||a — aj;|| < g^^^ , where az 
is the parallel translate of a along the geodesic on VF|j(y) from y to z. 

Finally, take 5q G (0,(5q] so small that for any z G W^^{zq) fl A we have (i($(z),zo) < ^"^^ 
d{z,'H'i^,z{^{z))) < S'q, where d is the standard distance in r(IR") = IR^". 

Now consider an arbitrary 6 G (0, Jq] and an arbitrary z G W'g'{zQ)r]A. Let b G E'^{z) be a unit 
vector tangent to A at z. Set x = ^(z), a' = d^{z) ■ b and y = T-Lto^zi^)- Then x G W|, (zq) n A, 
a' G E^{x) is a tangent vector to A at x (see Sect. 3) and y G W^, (z)nA. Moreover, since ||6|| = 1, 

it follows from \\d(^\\ < C (see Sect. 3) that ||a'|| < C. 

Next, the vector d = L^^ ^(x) . a' G E^{y) is tangent to A at y and by (6.3), \\d — a'\\ < 
Moreover, ||a|| < Ci||a'|| < CCi. Hence a = G E''{y) is a unit vector tangent to A at y, and 
using (6.2) and (6.3), we get 

\da{az^b)\ > \da{a,b)\ — \da{a — az,b)\ > --—r \da{d,b)\ 



8CC7i 



> 7^ [\da{a',b)\-\da{d-a',b)\] 



> 



1 



CCi 



\daid^z)-b,b)\-^ 



8CCi - ACCi 8CCi 



where = -g^^- This proves the assertion. ■ 

Prom Theorem 6.1 and Proposition 6.2 one derives the following. 

Theorem 6.3. Assume that the billiard flow (pt ■ A — > A satisfies the condition (P) on its 
non-wandering set A. Then for any Lipschitz real-valued function f on U the Ruelle transfer 
operators related to f are eventually contracting. 

Proof of Theorem 6.3. As mentioned in Sect. 1, the condition (P) implies that the map 

A 9 X I— >• E'^{x) is . Then by Proposition 6.2, (/>t satisfies the condition (ND) on A. Now 

applying Theorem 6.1 proves the assertion. ■ 

Results of this kind were first established by Dolgopyat ([DJ) for some Anosov flows (i.e. 
A = M, a compact Riemann manifold). His results apply to geodesic flows on any compact 
surface (for any /), and to transitive Anosov flows on compact Riemann manifolds with 
jointly non-integrable local stable and unstable foliations for the Sinai-Bowen-Ruelle potential 
/ = logdet(#r)|£;«. 

As one can see. Theorems 6.1 and 6.3 work for any potential. Theorem 6.3 generalizes the 
result in [St2j which deals with open billiard flows in the plane. 

It should be mentioned that Dolgopyat type estimates for pinched open billiard flows have 
already been used in [ PS1| . |PS2j and [PS3j to obtain some rather non-trivial results. The main 
result in |PS1| provides existence of an analytic continuations of the cut-off resolvent of the 
Dirichlet Laplacian in IR" \K va. a horizontal strip above the level of absolute convergence and 
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polynomial estimates for the norm of the cut-off resolvent in such a domain. The Dolgopyat 
type estimates for the open billiard flow in IR" \ K play a significant role in the proof. These 
estimates are also essential for the proof of the main result in jPS2) which deals with estimates 
of correlations for pairs of closed billiard trajectories for open billiards. Previous results of this 
kind were established in |PoS2| for geodesic flows on surfaces of negative curvature. Finally, in a 
very recent preprint [PS3j . using Theorem 6.3 a flne asymptotic was obtained for the number of 
closed billiard trajectories in A with primitive periods lying in exponentially shrinking intervals 
(x — e~^^',x + e~^^), 6 > 0, X ^ +oo. 

As in fStS], using Theorem 6.3 and an argument of Pollicott and Sharp [PoSl| . we get some 
rather significant consequences about the Ruelle zeta function C,{s) = Yi-yi^ ~ e"^^^'''^)"^ . Here 7 
runs over the set of primitive closed orbits of 0t : A — > A and ^(7) is the least period of 7. Let 
Ht be the topological entropy of (pt on A. 

Corollary 6.4. Under the assumptions in Theorem 6.3, the zeta function ((s) of the flow 
(pt '■ A — > A has an analytic and non-vanishing continuation in a half-plane Re(s) > cq for some 
Co < Ht except for a simple pole at s = hx- Moreover, there exists c £ (0, hx) such that 

7r(A) = #{7 : ^(7) < A} = n(e'^-^) + 0(e^^) 

r-^ du X 

as X ^ 00, where li(x) = / ~ as x —)• 00. 

J 2 log u log X 

As another consequence of Theorem 6.3 and the procedure described in ^ one gets exponen- 
tial decay of correlations for the flow (j)t : A — > A. 

Given a > denote by To (A) the set of Holder continuous functions with Holder exponent a 
and by \\h\\a the Holder constant of h £ J-a{A). 

Corollary 6.5. Under the assumptions in Theorem 6.3, let F he a Holder continuous function 
on A and let up he the Gihhs measure determined hy F on A. Assume in addition that the houndary 
of K is at least . Then for every a > there exist constants C = C (a) > and c = c{a) > 
such that 

j^A{x)B{(i)t{x)) dvF{x) - (_^^(^) dvF{x)^ (^J B{x) dvpix) 
for any two functions A, B £ Fa{A). 

One would expect that much stronger results could be established by using the techniques 
recently developed in |BKL] . [L] . |BG) . |GL) . [T] (see the references there, as well). Still, there 
are not very many results of this kind. In fact, for dimensions higher than two the author is 
not aware of any other results of this kind concerning billiard flows. What concerns billiards 
in general, bounds of correlation decay known so far concern mostly the corresponding discrete 
dynamical system (generated by the billiard ball map from boundary to boundary) - see |BSC| . 
[Y] and [Ch2j . See also [CJiZ] and the references there for some related results. Recently, a 
sub-exponential decay of correlations for Sinai billiards in the plane was established by Chernov 
( |Ch3] ). For open billiard flows in the plane exponential decay of correlations was proved in |St2] 
(as a consequence of the Dolgopyat type estimates established there). 
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